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Evaluation of the Effectiveness and Implementation of a 
Math Manipuiatives Project 

In March, 1989, the National Council of Teachers of Mathematics (NCTM) 
released its Curriculum and Evaluation Standards for School Mathematics. These 
standards were the result of three years ot planning, writ'r.g, and consensus-building 
to reflect solutions to national concerns about students’ mathematical 
performance.The NCTM Standards propose the development of enhanced 
mathematical power for all students by providing students with experiences to explore 
and reason, solve nonroutine problems, and develop personal self-confidence. Two 
assumptions about teaching and learning are central to this proposal: teachers are 
key figures in changing the ways in which mathematics is taught and learned in 
schools, and these changes require that teachers have long-term support and 
adequate resources. Specifically, designing effective experiences for students 
requires teachers with proficiency in using and helping students use, technology and 
other tools to pursue mathematical investigations and the ability to guide students in 
individual, small-group, and whole-class work. 

The use of mathematical tools such as manipuiatives has gained research 
support in the last two decades (Post, 1980; Raphael & Wahlstrom, 1989; Suydam & 
Higgins, 1977). Sowell (1989) concluded that mathematics achievement is increased 
by the long term use of manipuiatives and that student attitudes toward mathematics 
are improved when they are instructed with manipuiatives. Significant factors in 
mathematical achievement are the inclusion of manipuiatives and the expertise of the 
teachers using the manipuiatives. 

Project Design 

Based on the NCTM assumptions, research findings, and state and national 
efforts to improve mathematics education, Talladega College, in conjunction with high 
schools within a forty mile radius, developed a project to enrich algebra and geometry 
programs through the utilization of manipuiatives. Through participation in a week- 
long intensive training workshop and year-long followup activities, it was proposed 
that participants would a) enhance their teaching skills with the technological devices. 
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b) develop techniques for classroom implementation, and c) assess changes in 
students' achievement and attitudes. 

Instructional Techniques 

The workshop manual developed for the participants and the training sessions 
provided each participant with knowledge of the manipulatives approach to teaching 
mathematics. They gained knowledge from the printed materials and by observation 
of the methodologies utilized by the trainers. They also developed their own skills by 
formulating and presenting lessons as a culminating activity of the workshop. The 
participants received the kits, were trained, and had the opportunity to practice with the 
components during the workshop sessions. 

Followup activities 

The participants have provided inservice training for other teachers at the local 
level by conducting formal training sessions for 25 teachers, conducting informal 
training sessions on Saturday mornings, and by co-teaching situations in which 
implementation was demonstrated on a daily basis. 

All participants returned to Talladega for a 3-hour session in which specific 
strategies and problem areas were identified and discussed. The project staff 
provided feedback and suggestions for solutions for the identified problems. In 
addition, each teacher was observed by a project staff member, and practices were 
reviewed and discussed on-site. 

Assessment 

Dual assessment techniques were utilized : a) the effectiveness of the project 
instruction based on perceptions of the participants and observations by the project 
staff in an on-site immediate mods, and b) the effectiveness of the instruction based 
on degree and success of implementation and impact on students throughout the 
school year. The assessment component incorporated NCTM’s evaluation 
assumptions that the overall intent is to improve instruction and that teachers should 
be an mtegral part of that process. The standards emphasize that teachers should be 
encouraged and supported to engage in self-analysis and to work with colleagues in 
improving their teaching (NCTM, 1989). The participants utilized quantitative 
assessment instruments at the end of each session and completed a qualitative 



4 



4 



instrument to assist them in assessing impact on their teaching and their students' 
learning at the followup session. 

Teachers were asked to identify differences in their lesson plans and purposes 
with manipulatives and differences in their students' attitudes, participation, and 
performance. These differences were discussed and analyzed at the time of the local 
site visits during the second semester of implementation. , The process of identifying 
the differences was incorporated to cause the teachers to reflect on the impact of the 
manipulatives. 

Methodology 

Study Desig n 

The evaluation design utilized qualitative and quantitative methodology to 
determine the effectiveness and impact of the training with manipulatives. The final 
evaluation is based on a) review of the Eisenhower Proposal, b) participation in the 
planning process for the workshops with the project staff, c) observation of the 
methodology of the trainers and provisions for participant involvement in each of the 
workshop sessions, d) group discussion of classroom implementation strategies, e) 
observation of utilization of manipulatives in the participants' classrooms, and f) review 
of participants' self reports of student utilization and response to the manipulatives. 
Data collected from these resources were used to “assess the degree to which 
teachers are implementing the ideas and materials from the ... workshops" {Project 
Narrative, 1993, p. 14). 

Instruments 

The effectiveness of the instruction of the workshops was measured by the 
Evaluation of Eisenhower Workshop qualitative checklist (Appendix A) administered 
to the participants at the end of the week of each workshop session. Participants' 
perceptions of the usefulness of the provided manipulatives was assessed at the end 
of each session by the Participant Rating of Usefulness of Mathematics Manipulatives 
(Appendix B). 

The Evaluation of Eisenhower l/Vbrkshop qualitative checklist, created by one 
of the Project Directors, consisted of four open-ended questions which assessed a) 
quality of instruction, b) most valuable workshop components, c) least valuable 
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workshop components, and d) additional suggestions for improvement or future 
workshop topics. 

The Participant Rating of Usefulness of Mathematics Manipulatives, also 
created by one of the Project Directors, consisted of 23 items on a rating scale of 1-5 
(5 = most useful and 1 = least useful) and 3 open-ended questions assessing most 
useful material, least useful material, and recommendations for additionai resources. 

The degree of implementation and student response to the manipulatives were 
reported on the Math Manipulative Observation (Appendix C) and Math Manipulatives 
Workshop Followup (Appendix D). Evidence of utilization and preiiminary impact of the 
manipulatives as reflected in students' attitudes, class engagement, interaction with 
content, and academic performance was determined through the Math Manipulatives 
Observation, used by the project staff during on-site observations, and the Math 
Manipulatives Workshop Followup, a self-report by the participants. 

The Math Manipulatives Observation instrument was developed by the 
evaluator and designed to focus responses by the five observers during on-site 
observations. The evaluator provided training for the observers in the use of the 
instrument to enhance reliability in the recording of data. The observers recorded 
utilization by course and manipulative, student participation and attitudes toward the 
manipulatives, and interaction with the content. 

The Math Manipulatives Workshop Followup was designed to report courses in 
which manipulatives had been incorporated; differences in lesson plans; differences in 
students’ attitudes, participation, and achievement; and techniques utilized to train 
other teachers. 

Sample 

A total of 40 teachers representing 26 schools participated in the workshop and 
the evaluation processes. One of the original participants was reassigned as a 
counselor for the academic year, and one participant was not teaching courses with 
manipulatives during the observation period. Therefore, 38 teachers were observed, 
with a range of experience from 1 - 42 years, and a mean of 17.5 years. Classes 
observed Included; Math 7, Math 8, General Math, Pre-Algebra, Consumer Math, 
Technical Math, Algebra I, Algebra II, Geometry, Trigonometry, and Pre-Calculus. 
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Procedures 

These data were gathered by self-report by the participants and on-site 
observations by the project directors, the workshop presenters, and the outside 
evaluator. 

Treatment of the Data 

Analysis of the data was conducted by the project external evaluator. The 
responses to the Evaluation of Eisenhower Woi1<shop qualitative checklist were 
reviewed and coded. Categories of responses were identified which allowed 
summing of the responses according to teaching cKiracteristics, strategies employed, 
and specific manipulatives. Frequencies, mean ratings, and rank order were 
calculated for the 23 items of the Participant Rating of Usefulness of Mathematics 
Manipulatives . 

Data from the Math Manipulatives Observation instrument included calculating 
frequencies for utilization of manipulatives and coding and identifying categories for 
the open-ended observations. The self-report Math Manipulatives Workshop Followup 
instrument yielded qualitative data which were coded, categorized, and summarized. 

Results and Discussion 
Evaluation of Eisenhower Workshop 

Analysis of the codes and categories identified by the evaluator revealed 
assessment of the quality of instruction in the excellent (73%) to very good (18%) 
range. The participants were enthusiastic in their descriptions of the presenters as 
"knowledgeable," "caring" and "personable," and as being "well organized" and "on 
task." The responses to "the most valuable part" of the workshop included specific 
manipulatives as well strategies employed in the training. The TI-81 graphing 
calculator was identified by 73% of the participants and 40% specified "comparing 
ideas" with other teachers. The most frequently cited "least useful" part was "none," 
indicating that all parts were useful. Seven participants identified the dice as least 
useful. Additional comments included best workshop attended and suggestions for 
yearly followups and manipulatives for hands-on equations. 

Participant Rating of Usefulness of Mathematics Manipulatives 

As shown in Table 1, the means of the ratings range from 3.05 to 4.90, 
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indicating that all the manipulatives were rated in the useful category, with 19 of the 23 
items rated as 4 or better. Review of the manipulatives in rank order reveals that the 
calculators and geoboards were identified as the most useful items. 

The open-ended responses were consistent with the numerical ratings: the 
calculators were identified as the most useful and the geoboards were the second 
most cited manipulative. The dice and the spinners were identified as least useful. It 
is recognized that the participants’ ratings of the usefulness are determined by the 
subject matter and student response; therefore, the homogeneity of responses 
indicates strong support for the inclusion of calculators and geoboards in the 
mathematics classroom. 

Math Manipulatives Observation 

Attitudes . Observers concluded that students enjoyed using the manipulatives 
as expressed through their verbal and nonverbal behaviors. They were involved with 
the manipulatives and the tasks associated with them. Students exhibited confidence, 
eagerness, a desire for other experiences, and an inquisitive approach in their use cf 
the manipulatives. 

Participation . The “on task" involvement was very high, 100% in most classes. 
Observers noted an enhanced willingness to respond to questions and several 
students extended discovery beyond the assignment. 

Interaction . Observers noted that the students seemed to comprehend tasks 
with accuracy, employed discovery and problem solving strategies, were anxious to 
share their discoveries and solutions, engaged in lively student/student interaction 
related to the content, and exhibited an excitement about learning. 

Math Manipulatives Workshop Followup 

Planning . Teachers indicated that more time was needed for planning and 
more class time was needed when manipulatives are used, but student participation is 
enhanced. The manipulatives supplement and reinforce established concepts and 
allow them to utilize discovery groups to a greater extent. 

Attitudes . The teachers reported that the students enjoy and are more 
interested in assignments (and even express excitement) when manipulatives are 
used. They are more active in class, develop self-confidence in their math skills, and 
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are willing to experiment. 

Participation . Enhanced participation and a desire to participate were reported 
by the teachers. Some teachers noted more student cooperation and enhanced 
student/teacher interaction. Manipulatives were the catalyst for participation by some 
who normally are nonparticipants. 

Performance . Most teachers noted that the intervention time was too short and 
no pretest data were available to detect differences in standardized test scores, but 
other improvements were noted. Improvement on local tests was noted by 46% of the 
teachers, and others noted that the students exhibited understanding of the concepts 
to a greater degree. Two teachers noted that below average students’ scores 
improved, and other teachers noted improvements in Algebra and General Math 
scores. 

Inservice training . All teachers have been involved in some type of activity to 
train other teachers in their schools or systems. They have conducted formal training 
sessions for 12-25 teachers, conducted informal training sessions after school and on 
Saturday mornings for one to nine teachers, shared ideas including developing lesson 
plans with one to five teachers, and co-taught in which implementation was 
demonstrated on a daily basis. 

In conclusion, the project was successful in addressing the stated objectives of 
the proposal through the techniques of well organized workshop training sessions, 
followup reporting and evaluation, and extended local training. The manipulatives are 
being utilized extensively, and student attitudes, participation, and performance have 
been enhanced. 
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APPENDIX A 



EVALUATION OF EISENHOWER WORKSHOP CONDUCTED BY 
TALLADEGA COLLEGE DURING SUMMER 1993 



Please comment on the quality of mstruction provided by the project 
staff 



Identify the most valuable parts of the workshop. 



Identify the least valuable parts of the workshop. 



Please make any additional comments on ways to improve future 
workshops and/or give additional topics to be covered. 



APPENDIX B 

PARTICIPANT RATING OF USEFULNESS OF MATHEMATICS 

MANIPULATIVES 

Please rate on a scale from 1-5, with 1 being least useful and 5 being 
most useful, your evaluation of each component of the Mathematics 
Manipulatives Kit 

1 Set Dice (6 Pairs in 3 Colors) 

5 Sets/6 Polyhedra Dice 

4 Sets/4 Overhead Spinners 

1 Set/250 Pattern Blocks 

1 Set/49 Overhead Pattern Blocks 

1 Set/200 Transparent Circular Counters 

1 Set/250 Overhead Counters 

1 Set/100 Color Cubes 

1 Set/36 Overhead Attribute Blocks 

15 Wooden Geoboards 

1 Package Rubber Bands in 3 Colors 

1 Overhead Geoboards 

1 Set Overhead Fraction Bars 

15 Sets/32 Algebra Tiles 

1 Set/70 Overhead Algebra Tiles 

15 Triman Circle Protractors 

15 Triman Classmate Compasses 

1 Overhead Triman Compass 

1 Sage Kit Transparent Geometric Models 

15 Calculators TI-34 

1 Overhead Calculator for TI-34 

5 Calculators TI-81 (graphing) 

1 Overhead Calculator for TI-81 (graphing) 
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1. Which of the materials did you find to be most useful? 

2. Which of the materials did you find to be least useful? 

3. What additional resources would you recommend for teachers of 
grades 7-12 mathematics? 

PLEASE USE THE SPACE BELOW FOR THESE AND/OR 
ADDITIONAL COMMENTS. 



r 



Appendix C 

MATH MANIPULATIVES OBSERVATION 

N AME SCHOOL 

SYSTEM COURSE 

DATE TIME 

PURPOSE OF LESSON 



MANIPULATIVES UTILIZED. 



HOW MANIPULATIVES WERE UTILIZED (DEMONSTRATION, LARGE 
GROUP, SMALL GROUP, INDIVIDUAL HANDS-ON) 



STUDENTS' ATTITUDES TOWARD USE OF MANIPULATIVES : 



STUDENTS’ CLASS PARTICIPATION WHEN MANIPULATIVES ARE 
UTILIZED: 



INTERACTION OF LEARNERS WITH CONTENT DUE TO USE OF 
MANIPULATIVES 



REVIEWER. 
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Appendix D 

MATH MANIPULATIVES WORKSHOP FOLLOWUP 

NAME #YRS. TEACHING MATH 

COURSES IN WHICH MANIPULATIVES/TECHNIQUES FROM SUMMER 
WORKSHOP HAVE BEEN INCORPORATED: 



HOW ARE LESSON PLANS/PURPOSES DIFFERENT WITH 
MANIPULATIVES? 



DESCRIBE ANY DIFFERENCES YOU HAVE OBSERVED IN STUDENTS’ 
ATTITUDES WHEN MANIPULATIVES ARE UTILIZED: 



DESCRIBE ANY DIFFERENCES YOU HAVE OBSERVED IN STUDENTS’ CLASS 
PARTICIPATION WHEN MANIPULATIVES ARE UTILIZED: 



DESCRIBE ANY DIFFERENCES YOU HAVE OBSERVED IN STUDENTS’ 
PERFORMANCE ON YOUR TESTS OR STANDARDIZED TESTS WHEN 
MANIPULATIVES ARE UTBJZED: 



HAVE YOU PROVIDED ANY TRAINING OF OTHER TEACHERS IN YOUR SCHOOL 

OR SYSTEM? IF YES, DESCRIBE NUMBER OF TEACHERS AND NATURE 

OF TRAINING. 



INDICATE WHE^J AND HOW IT IS PLANNED; 



IF NO, 
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